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This  report  responds  to  your  request  that  we  review  the  software 
development  capabilities  of  the  organizations  responsible  for  developing  a 
critical  enhancement  to  the  National  Weather  Service’s  (nws)  Advanced 
Weather  Interactive  Processing  System  (awips),  known  as  the  awips 
Forecast  Preparation  System  (afps).  awips  is  an  information  processing 
system  to  support  forecasters  in  acquiring,  analyzing,  and  disseminating 
weather  data  from  various  sources.  The  afps  enhancement  is  to  automate 
additional  functions  currently  performed  by  the  forecasters,  thereby 
streamliiung  the  forecast  process  and  improving  forecaster  productivity. 
The  National  Oceanic  and  Atmospheric  Administration  (noaa),  of  which 
nws  is  apart,  is  jointly  developing  afps  with  nws  through  the  respective 
laboratories.  Their  plan  is  to  first  define  an  afps  software  specification 
through  a  series  of  prototype  systems  and  then  develop  afps 
production-quality  software1  for  direct  integration  into  awips. 

Our  objective  was  to  determine  whether  the  software  development 
processes  of  noaa’s  Forecast  Systems  Laboratory  (fsl)  and  nws’ 

Techniques  Development  Laboratory  (tdl)  are  adequate  to  support 
(  )  afps  software  prototyping  and  (2)  afps  production-quality  software 
evelopment.  As  a  rule,  the  software  development  processes  needed  to 
wnte  production-quality  software  are  much  more  rigorous,  disciplined 
and  formal  than  those  used  for  software  prototyping.  The  laboratories’’ 
software  development  processes  in  question  are  software  requirements 
management,  project  planning,  quality  assurance,  configuration 


r  SOftware  15  software  **  G)  satisfies  its  specified  functional,  performance  and 
operational  requirements  and  thus  is  ready  for  day-to-day  use  and  (2)  has  effective  documentation  ffor 
example,  programmers’  manuals,  users’  manuals,  comments  on  code  Stog  p^d^i“i 
etc.)  to  support  system  operations  and  maintenance.  procedures  ana  results, 
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management,  and  tracking  and  oversight.  A  detailed  explanation  of  these 
processes  is  presented  in  appendix  I. 

Results  in  Brief 

'  The  software  development  processes  that  fsl  and  tdl  have  in  place  and 
are  following  in  developing  afps  are  adequate  to  support  nws’  near-term 
afps  development  activities— prototyping  for  the  purpose  of  defining  afps 
requirements  and  preparing  a  software  specification.  However,  the 
laboratories’  processes  are  not  adequate  to  achieve  nws’  ultimate 
objective — developing  production-quality  afps  code  that  nws  can  give  to 
the  awips  contractor  for  direct  integration  into  awips.  Unless  the 
laboratories  introduce  formality,  rigor,  and  discipline  into  their  software 
development  processes  before  they  begin  writing  production-quality  code, 
afps  and  awips  will  likely  perform  poorly,  be  delivered  late,  and  cost 
considerably  more  than  planned.  Officials  for  both  laboratories 
acknowledged  their  respective  process  limitations  relative  to  developing 
production-quality  code,  and  stated  that  needed  improvements  are 
planned. 

Background 

"  Since  the  early  1980s,  nws  has  been  modernizing  its  observing,  information 
processing,  and  communications  systems  to  improve  the  accuracy, 
timeliness,  and  efficiency  of  weather  forecasts  and  warnings.  The 
modernization  includes  four  new  major  systems— awips,  the  Next 
Generation  Geostationary  Operational  Environmental  Satellites,  Next 
Generation  Weather  Radars,  and  Automated  Surface  Observing  Systems. 
The  Department  of  Commerce’s  Deputy  Under  Secretary  for  Oceans  and 
Atmosphere  is  responsible  for  the  modernization. 

AWIPS  Description  and 
Status 

As  discussed  in  our  March  1994  report,2  the  heart  of  the  modernization  is 
awips.  awips  will  allow  forecasters  to  integrate,  manipulate,  and  analyze  the 
vast  amounts  of  data  that  are  expected  to  be  available  from  the  new  or 
improved  observing  and  information  processing  systems. 

noaa  awarded  the  awips  contract  to  the  Planning  and  Research 

Corporation  (prc)  in  December  1992  with  an  atypical  arrangement  in 
which  nws  would  supply  prc  with  certain  specifications  in  the  form  of 
existing  and  to-be-developed  code.  Under  this  arrangement,  prc  would 
then  decide,  with  noaa’s  and  nws’  approval,  whether  the  code  was  of 

2Weather  Forecasting:  Systems  Architecture  Needed  for  National  Weather  Service  Modernization 
(GAO/AIMD-94-28,  March  11, 1994). 
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sufficient  quality  to  be  directly  incoiporated  into  awips  or  whether  it 
required  rewriting. 

nws  is  now  in  the  process  of  restructuring  the  awips  program  and 
renegotiating  the  awips  contract  with  prc.  These  changes  are  in  response 
to  awips  design  problems  and  program  delays.  Although  all  the  details 
surrounding  program  and  contract  changes  have  not  been  defined,  nws 
officials  told  us  that  the  restructured  program  will,  among  other  things, 
assign  responsibility  for  developing  awips  application  software  to  the 
government.  This  government-developed  application  software  will  then  be 
provided  to  prc,  which  will  be  responsible  for  providing  the  awips  platform 
(that  is,  hardware  and  systems  software  environment),  integrating  the 
applications  with  the  platforms,  and  ensuring  overall  system  quality. 
According  to  these  officials,  prc  will  be  contractually  required  to  use  the 
government-furnished  code,  including  the  afps  code,  as  delivered,  and 
integrate  it  into  awips  (that  is,  treating  it  as  government-furnished 
equipment).  However,  the  specific  contract  terms  that  define  such  things 
as  system  quality  and  responsibility  for  poor  system  performance  have  not 
been  specified. 


AFPS  Description  and  afps  is  part  of  the  NWS  to-be-provided  application  software.  Estimates  of  its 

btatus  size  ranSe  100,000  to  250,000  lines  of  code.  The  system  is  expected  to 

streamline  the  forecast  process  and  improve  forecaster  productivity 
through  automated: 

Forecast  visualization  -  The  process  of  forming  an  on-screen  graphical 
forecast  image  based  on  observations,  direct  model  input,  numerical  and 

statistical  forecasts,  climatology,  and  other  data,  as  well  as  experience  and 
training. 

Graphical  forecast  editing  -  The  process  of  graphically  entering  and 
revising  watch,  warning,  and  advisory  information  used  in  the  original 
visualized  forecast. 

Text  generation  -  The  process  of  automatically  producing  text  based  on 
the  graphical  representation  of  the  forecast,  thus  relieving  the  forecasters 
of  repetitive  and  time-consuming  typing  responsibilities. 

The  laboratories  are  currently  defining  and  validating  afps  requirements 
through  the  use  of  prototyping  techniques.  Prototyping  is  the  iterative 
process  of  quickly  coding,  evaluating,  and  refining  less  than  complete 
versions  of  a  system.  As  such,  a  system  prototype  is  not  intended  to  be  an 
end  product  or  final  system;  instead,  the  prototype  is  a  learning  tool  or  a 
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series  of  learning  tools.  Prototyping  may  be  undertaken  to  conduct 
research  (for  example,  to  prove  a  concept,  determine  a  project  s 
feasibility,  or  define  system  requirements),  and  thus,  once  the  goals  are 
achieved,  the  prototype  can  be  discarded.  However,  prototypes  can  also 
be  retained  and  used  as  a  foundation  for  enhancement  or  be  repackaged 
as  a  final  product. 

To  date,  the  laboratories  have  developed  four  afps  iterations.  According  to 
laboratory  officials,  their  initial  afps  prototype  was  a  “throw  away.  The 
next  prototype,  however,  formed  the  foundation  on  which  additional 
system  functions  and  features  have  and  will  continue  to  be  added  in 
building  increasingly  more  capable  iterations.  The  laboratories  plan  is  to 
continue  prototyping  until  afps  is  refined  to  the  point  that  it  can  be  placed 
in  an  nws  field  office  for  validation  by  weather  forecasters  in  an 
operational  setting.  On  the  basis  of  what  is  learned  in  the  field  office,  afps 
is  to  be  further  refined  before  providing  it  to  nws  for  additional  testing  and 
refinement,  nws  is  then  to  deliver  the  code  to  prc  for  integration  into  awips. 
According  to  laboratory,  noaa,  and  nws  officials,  the  current  plan  is  to 
deliver  code  that  requires  no  rewrite  before  it  is  integrated  into  awips.  The 
officials  emphasized  that  their  goal  is  to  develop  production-quality  code. 


Importance  of  Software 
Development  Discipline 


Software  development  has  proven  to  be  the  “Achilles  heel”  of  many  system 
development  projects.  Frequently  these  projects  are  delivered  late,  exceed 
budgets,  and  perform  poorly  because  the  software  development  was  not 
guided  by  disciplined  engineering  processes. 

The  rigor  and  formality  of  the  processes  needed  to  successfully  develop 
software  are  determined  by  the  desired  outcome.  If  the  goal  is  to  develop 
one  or  more  prototypes  that  are  to  be  used  as  learning  tools  and  then 
discarded,  formal  software  processes  and  extensive  documentation  are 
unnecessary.  However,  if  the  desired  outcome  is  production-quality  code 
for  the  final  system,  more  rigorous  and  stringent  software  engineering 
processes  should  guide  the  development  effort.3 


Scope  and 
Methodology 


To  determine  the  laboratories’  development  capabilities,  we  reviewed  key 
software  development  documents,  including  the  afps  development  plan, 
software  development  and  documentation  guidelines,  summaries  of 
prototyping  cycles,  and  quarterly  reports.  In  addition,  we  interviewed  noaa 


’Rigorous  software  engineering  processes  are  those  that  are  structured,  well  documented,  and 
systematically  implemented  and  monitored.  See  appendix  H  for  specific  examples  of  such  processes. 
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and  nws  officials  at  tdl  and  fsl  about  their  processes.  To  identify  the 
software  development  processes  that  can  reasonably  be  expected  to  be  in 
place  when  prototyping  a  system  versus  developing  production-quality 
software,  we  researched  relevant  literature  and  government  and  industry 
standards,  reviewed  the  Capability  Maturity  Model  developed  by  Carnegie 
Mellon  University’s  Software  Engineering  Institute  (sei),  and  interviewed 
sei  officials,  sei’s  Capability  Maturity  Model  is  a  tool  for  assessing 
organizations  ability  to  develop  software  in  accordance  with  modem 
software  engineering  methods.  This  tool  focuses  on  the  maturity  of  certain 
software  development  processes.  The  processes  applicable  to  the 
laboratories  are  (1)  software  requirements  management,  (2)  project 
planning,  (3)  quality  assurance,  (4)  configuration  management,  and 
(5)  tracking  and  oversight. 

We  performed  our  work  at  noaa  and  nws  headquarters  and  at  nws’  tdl  in 
S^ver  Spring,  Maryland;  noaa’s  fsl  in  Boulder,  Colorado;  and  sei  in 
Pittsburgh,  Pennsylvania.  Our  work  was  performed  from  May  1994 
through  October  1994,  in  accordance  with  generally  accepted  government 
auditing  standards. 


Laboratories’ 
Processes  Are 
Adequate  for  AFPS 
Software  Prototyping 


The  processes  that  the  laboratories  have  in  place  for  software 
requirements  management,  project  planning,  quality  assurance, 
configuration  management,  and  tracking  and  oversight  are  adequate  for 
afps  prototyping.  Laboratory  activities  in  each  of  the  five  process  areas 
that  we  reviewed  satisfied  the  level  of  structure  and  documentation 
advocated  by  sei  for  determining  requirements  and  defining  software 
specifications  through  system  prototyping.  In  particular,  sei  officials  stated 
that  prototyping  is  an  investigative  process  that  typically  requires  flexible 
processes  so  as  not  to  overwhelm  the  developers’  creativity.  Accordingly, 
fsl  has  adopted  a  process  known  as  the  spiral  model  of  software 
development  to  define  requirements.  This  process  involves  iterative  builds, 
each  enhancing  the  previous  one.  tdl  and  fsl  have  also  augmented  the  use 
of  this  model  with  some  software  development  formality,  such  as  the  use 
of  documented  software  coding  guidelines  and  a  documented  software 
development  plan. 
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Laboratories’ 
Processes  Are  Not 
Adequate  for 
Developing  AFPS 
Production-quality 
Software 


The  laboratories  do  not,  however,  have  in  place  the  software  development 
processes  that  are  needed  to  develop  high-quality,  production  code. 

Instead,  their  processes  are  largely  informal,  relying  more  on  the 
capabilities  of  laboratory  staff  rather  than  clearly  defined  and  documented 
processes.  While  certain  laboratory  process  areas  (for  example, 

requirements  management)  are  stronger  than  others  (for  example, 

software  quality  assurance),  none  possess  the  full  complement  of  activities 
that  the  sei  Capability  Maturity  Model  advocates.  Specific  examples  of 
where  the  laboratories  are  deficient  in  each  process  area  are  described 
below.  Additional  examples  are  provided  in  appendix  H 

.  Requirements  management:  Organizations  developing  production-quality 
software  should  have  a  software  engineering  group  that  reviews  and 
agrees  to  requirements  before  code  is  written  to  incorporate  these 
requirements  into  the  software.  Neither  tdl  nor  fsl  have  software 

engineering  groups.  ,  .  .  . 

.  Project  planning:  The  software  development  plan  is  the  culmination  ot 
software  project  planning.  To  the  laboratories’  credit,  they  have  an  afps 
software  development  plan  that  defines  the  project’s  purpose,  goals, 
organizational  development  responsibilities,  software  development 
methodology,  and  software  development  schedules.  However,  their  plan 
lacks  other  important  elements,  including  software  verification  and 
validation  provisions  and  software  metrics. 

.  Quality  assurance:  The  software  quality  assurance  plan  is  the  centerpiece 
of  an  effective  quality  assurance  program.  This  plan  defines  the  activities 
necessary  to  ensure  that  software  development  processes  and  products 
conform  to  applicable  requirements  and  standards.  In  addition,  an 
independent  software  quality  assurance  group  should  review  and  audit  the 
software  engineering  activities  to  ensure  compliance.  The  laboratones  do 
not  have  a  software  quality  assurance  plan  or  group. 

.  Configuration  management:  A  software  configuration  management  plan 
should  exist  that  clearly  defines  the  procedures  for  identifying,  accounting 
for,  and  reporting  on  changes  to  software  items  that  are  under 
configuration  control.  Neither  laboratory  has  a  software  configuration 
management  plan. 

.  Tracking  and  oversight:  To  ensure  that  a  software  development  project  is 
proceeding  as  planned,  formal  reviews  to  communicate  accomplishments 
and  results  of  the  project  software  engineering  should  be  conducted  at 
selected  project  milestones.  The  laboratories  do  not  conduct  such  reviews. 

fsl  and  tdl  officials  agreed  that  their  software  development  processes  are 
not  adequate  for  production-quality  development  activities.  They  stated 


Page  6 


GAO/AIMD-95-24  Weather  Service  Software  Development 


B-259114 

that  actions  are  planned  to  improve  the  maturity  of  these  processes.  For 
example,  fsl  plans  to  hire  an  individual  to  establish  and  manage  an 
independent  quality  assurance  program,  including  development  and 
implementation  of  a  quality  assurance  plan.  Also,  fsl  plans  to  improve  its 
requirements  management  process  by  (1)  documenting  the  process, 

(2)  developing  a  technique  for  mapping  system  requirements  to  the  system 
and  software  designs,  and  (3)  documenting  changes  to  the  requirements 
baseline. 

Conclusions 

Although  fsl  and  tdl  have  adequate  software  development  processes  for 
defining  afps  requirements  via  prototyping,  neither  has  adequate  processes 
for  developing  production-quality  code,  which  is  nws’  ultimate  goal  on 
afps.  Without  these  processes,  nws  is  exposing  afps,  as  well  as  awips,  to 
unnecessary  cost,  schedule,  and  performance  risks. 

Up- - — _ _ _ 

Recommendations 

In  light  of  nws’  plan  to  provide  the  awips  contractor  with 
production-quality  software  for  direct  integration  into  awips,  we 
recommend  that  the  Secretary  of  Commerce  direct  the  Deputy  Under 
Secretary  for  Oceans  and  Atmosphere  to  have  fsl  and  tdl  strengthen  then- 
software  development  processes  for  requirements  management,  project 
planning,  quality  assurance,  configuration  management,  and  tracking  and 
oversight  before  beginning  development  of  any  production-quality  code. 

The  crucial  processes  are  outlined  in  appendix  n  of  this  report. 

We  discussed  the  contents  of  this  report  with  tdl  and  fsl  officials,  the 
awips  Technical  Director,  and  the  nws  Modernization  Systems  Manager. 
These  officials  generally  agreed  with  the  information  presented.  We  have 
incorporated  their  comments  where  appropriate. 

We  are  sending  copies  of  this  report  to  the  Secretary  of  Commerce,  the 
Director  of  the  Office  of  Management  and  Budget,  and  interested 
congressional  committees.  Copies  will  also  be  made  available  to  others 
upon  request. 
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Please  call  me  at  (202)  512-6253  if  you  or  your  staff  have  any  questions 
concerning  this  report.  Other  major  contributors  are  listed  in  appendix  HI. 


Joel  C.  Willemssen 

Director,  Information  Resources 

Management/Resources,  Community, 
and  Economic  Development 
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Software  Development  Process  Areas 


Table  1.1  provides  the  software  development  process  areas  we  evaluated 
and  their  definitions. 


Table  1.1 :  Software  Development 
Process  Areas 


Software  requirements 
management 

This  process  involves  establishing  and  maintaining  an 
agreement  with  the  customer  for  both  the  technical  and 
nontechnical  requirements  for  the  software  project.  This 
agreement  forms  the  basis  for  estimating,  planning, 
performing,  and  tracking  the  software  project’s  activities 
throughout  the  software  life  cycle. 

Software  project  planning 

This  process  is  a  subset  of  the  overall  project  planning.  It 
involves  defining  each  major  software  project  task, 
estimating  the  time  and  resources  required  to  accomplish 
it,  and  tracking  and  controlling  actual  software  production 
against  these  and  other  production  goais.  The 
centerpiece  of  software  project  planning  is  the  software 
development  plan. 

Software  quality  assurance 

This  process  is  the  planned  and  systematic  set  of  actions 
necessary  to  provide  sufficient  confidence  that  the 
software  product  conforms  to  established  requirements. 
Effective  quality  assurance  should  be  (1)  managed  by  an 
organization  independent  of  the  organizations  managing 
the  project  and  developing  the  software  products, 

(2)  managed  by  an  organization  that  has  the  authority  to 
establish  and  enforce  software  quality  standards  and 
procedures,  (3)  based  on  predetermined  software 
metrics,  and  (4)  managed  by  documented  processes  that 
are  shared  among  parties  participating  in  the  project. 

Software  configuration 
management 

This  process  is  the  means  by  which  changes  to  software 
products  are  controlled.  It  includes  identification  of 
software  products  to  be  controlled,  accounting  for 
changes  to  these  controlled  products,  and  reporting  on 
the  status  of  these  products. 

Software  tracking  and 
oversight 

This  process  provides  insight  into  project  progress  and 
provides  a  basis  for  reporting  on  project  status.  It  is 
accomplished  by  measuring  ongoing  software 
development  activities  and  comparing  the  measurements 
against  documented  estimates,  commitments,  plans,  and 
industry  norms. 
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Comparison  of  Crucial  Production  Software 
Development  Activities  With  Those  at  TDL 
and  FSL 


This  appendix  identifies  the  activities1  that  are  expected  of  leading 
software  development  organizations  and  contrasts  these  with  the  activities 
cun-ently  performed  at  tdl  and  fsl.  While  this  is  not  a  comprehensive  list 
of  activities,  it  highlights  the  most  crucial  activities  desired  in  each  of  the 
software  development  process  areas. 


Table  11.1:  Software  Requirements 
Management  Process 


Crucial  activities 

The  requirements  are  documented  in  a 
consistent  format  and  are  clearly  stated, 
verifiable,  and  testable. 


The  software  engineering  group  reviews  and 
agrees  to  the  requirements  before  they  are 
incorporated  into  the  software  efforts. 


TDL  and  FSL  activities 

The  requirements  are  documented  with 
each  prototype  iteration.  Each  iteration 
more  clearly  defines  the  baseline 
requirements. 

A  formal  software  engineering  group  does 
not  exist. 


The  requirements  form  the  basis  for  the 
software  plans,  products,  and  activities. 


Changes  to  the  requirements  are 
appropriately  reviewed  and  incorporated 
into  the  software  efforts. 


The  AFPS  requirements  are  still  being 
developed,  and  thus  such  documents  and 
activities  do  not  yet  exist. 

Requirements  changes  are  reviewed  by  a 
technical  coordinator  and  are  incorporated 
into  the  next  prototype  iteration.  _ 


lrThe  activities  were  derived  from  SEI’s  Capability  Maturity  Model. 
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Appendix  II 

Comparison  of  Crucial  Production  Software 
Development  Activities  With  Those  at  TDL 
and  FSL 


Table  11.2  Software  Project  Planning 
Process 


Crucial  activities _ 

The  software  engineering  group  is  an  active 
participant  in  proposing  and  planning  the 
project  throughout  its  life. 

Software  planning  is  initiated  in  the  early 
stages  of,  and  in  parallel  with,  overall  project 
planning. 

Software  planning  data  are  recorded  for  use 
by  the  project. 

Senior  management  reviews  and  approves 
all  commitments  made  to  individuals  and 
groups  external  to  the  organization. 

A  software  life-cycle  model  with 
predetermined  stages  of  manageable  size  is 
identified  or  defined. 

The  project’s  software  development  plan  is 
developed  according  to  a  documented 
procedure  and  addresses  all  software 
activities. 


TDL  and  FSL  activities _ 

There  is  no  formal  software  engineering 
group,  but  both  TDL  and  FSL 
programmers  participate  in  the  project’s 
planning  activities. 

The  AFPS  software  development  plan  was 
developed  in  conjunction  with  the  AFPS 
concept  document. 

This  activity  does  not  exist. 


No  arrangements  with  external 
organizations  exist. 


A  software  life-cycle  model  does  not  exist. 


The  AFPS  software  development  plan  is 
documented  in  the  AFPS  concept 
document  and  addresses  most  software 
activities. 


Software  products  and  software  process 
specifications  that  are  needed  to  establish 
and  maintain  stability  of  the  software 
activities  are  explicitly  identified  as 
controlled  project  baseline  items. 

Estimates  for  the  size  of  the  software 
products,  the  software  development 
resources  and  costs,  the  critical  target 
computer  resources,  and  the  software 
schedule  are  derived  according  to  a 
documented  procedure. 

The  software  technical,  cost,  resource,  and 
schedule  risks  are  assessed  and 
documented. 


Software  product  and  process 
specifications  (for  example,  design 
documents,  programming  guidelines)  exist 
but  are  not  identified  as  controlled  project 
baseline  items. 

A  documented  procedure  to  derive  these 
items  does  not  exist. 


Such  risks  are  identified  and  assessed 
informally;  however,  they  are  not 
documented. 
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Table  11.3:  Software  Quality  Assurance 
Process 


Appendix  II 

Comparison  of  Crucial  Production  Software 
Development  Activities  With  Those  at  TDL 
and  FSL 


Crucial  activities _ TDL  and  FSL  activities 

A  software  quality  assurance  (SQA)  plan  is  An  SQA  plan  does  not  exist. 

prepared  for  each  software  project 

according  to  a  documented  procedure,  and 

SQA  activities  are  performed  in  accordance 

with  the  SQA  plan. 

The  SQA  group  participates  in  the  An  SQA  group  does  not  exist. 

preparation,  review,  and  approval  of  the 

project's  software  development  plan, 

process  specifications,  standards,  and 

procedures. 


The  SQA  group  reviews  and  audits  the 
software  engineering  activities  to  ensure 
process  compliance. 


The  SQA  group  reviews  representative 
samples  of  deliverable  and  designated 
nondeliverable  software  products  to  ensure 
compliance  with  the  designated  process 
requirements. 

The  SQA  group  regularly  reports  its  reviews 
and  audits  to  the  software  engineering  staff 
and  managers. 

Deviations  identified  in  the  software 
engineering  activities  are  documented  and 
handled  according  to  a  documented 
procedure. 


The  FSL  Technical  Coordinator  reviews 
code  and  conducts  code  waik-throughs. 
TDL  staff  have  peer  code  reviews  and 
walk-throughs.  However,  a  separate  SQA 
group  does  not  exist. 

Such  activities  do  not  occur. 


Such  activities  do  not  occur. 


Deviations  identified  as  a  result  of  the  FSL 
Technical  Coordinator  and  TDL  peer 
reviews  are  addressed  but  not 
documented  nor  handled  according  to 
documented  procedures. 
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Appendix  II 

Comparison  of  Crucial  Production  Software 
Development  Activities  With  Those  at  TDL 
and  FSL 


Table  11.4:  Software  Configuration 
Management  Process 


Crucial  activities 


TDL  and  FSL  activities 


A  documented  software  configuration 
management  (SCM)  plan  exists  and  is  used 
as  the  basis  for  performing  SCM  activities. 

A  configuration  management  library  system 
is  established  as  a  repository  for  the 
software  baselines. 


The  software  engineering  products  and 
process  specifications  to  be  placed  under 
configuration  management  are  identified. 

A  documented  procedure  is  followed  for 
initiating,  recording,  reviewing,  approving, 
and  tracking  change  requests  and  trouble 
reports  for  all  configuration  items. 

A  documented  procedure  is  followed  to 
create  and  control  the  release  of  software 
baseline  products. 

A  documented  procedure  is  followed  to 
record  the  status  of  configuration  items  and 
change  requests,  and  to  control  changes  to 
configuration  items. _ 


An  SCM  plan  does  not  exist. 


A  configuration  management  library 
system  for  managing  the  software  baseline 
does  not  exist.  However,  the  laboratories 
use  documented,  automated  tools  for 
configuration  control  of  the  software 
products. 

Such  products  and  processes  have  not 
been  identified  for  placement  under 
configuration  control. 

A  documented  procedure  does  not  exist 
for  tracking  and  controlling  change 
requests  and  trouble  reports. 

A  documented  procedure  does  not  exist  to 
control  software  baselines. 


Such  documented  procedures  do  not  exist. 
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Appendix  II 

Comparison  of  Crucial  Production  Software 
Development  Activities  With  Those  at  TDL 
and  FSL 


Table  11.5:  Software  Tracking  and 
Oversight  Process 


Crucial  activities 

A  documented  software  development  plan 
is  used  for  tracking  the  software  activities 
and  communicating  status. 


Senior  management  reviews  and  approves 
all  commitments  to  individuals  and  groups 
external  to  the  organization. 

Approved  changes  to  software 
commitments  or  commitments  affecting  the 
software  activities  are  explicitly 
communicated  to  the  staff  and  managers  of 
the  software  engineering  group. 

The  project’s  size,  costs,  critical  target 
computer  resources,  software  schedule, 
and  software  engineering  activities  are 
tracked  and  corrective  actions  are  taken. 

The  software  technical,  cost,  resource,  and 
schedule  risks  are  tracked  throughout  the 
life  of  the  project. 

Actual  measured  data  and  replanning  data 
for  the  software  project  tracking  activities 
are  recorded  for  use  by  the  software 
engineering  staff  and  managers. 

The  software  engineering  staff  and 
managers  conduct  regular  reviews  to  track 
technical  progress,  plans,  performance,  and 
issues  against  software  development  plans. 

Formal  reviews  to  address  the 
accomplishments  and  results  of  project 
software  engineering  are  conducted  at 
selected  project  milestones. _ 


TDL  and  FSL  activities 

A  software  development  plan  is  not  used 
for  tracking  the  software  activities; 
however,  such  activities  are  tracked  and 
communicated  through  quarterly  reports. 

No  arrangements  with  external 
organizations  exist. 


Software  changes  are  communicated 
informally  between  laboratory  officials. 


These  activities  are  not  tracked  by  any 
formal  means. 


These  risks  are  not  tracked  by  any  formal 
means. 


Such  activities  do  not  occur. 


Such  regular  reviews  to  track  such  items 
against  software  development  plans  do  not 
occur. 


Formal  reviews  are  not  conducted. 
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